Tempered auto-adjusting, image-editing operation

ABSTRACT

Some embodiments provide a novel method for tempering an adjustment of an image to account for prior adjustments to the image. The adjustment in some embodiments is an automatic exposure adjustment. The method performs an operation for a first adjustment on a first set of parameters (e.g., saturation, sharpness, luminance). The method compares the first set of parameters to a second set of parameters to produce a third set of parameters that expresses the difference between the first adjustment and a second adjustment. The method performs a third operation to produce an adjusted image. The first set of parameters quantify a set of prior adjustments to the image by an image capturing device when the image was captured in some embodiments. The second set of parameters is a set of target parameters. The third set of parameters specify the tempered adjustment of the image.

CLAIM OF BENEFIT TO PRIOR APPLICATIONS

This application claims benefit to U.S. Provisional Patent Application61/657,794, entitled “Tempered Auto-adjusting, Image-editing Operation,”filed Jun. 10, 2012. The U.S. Provisional Patent Application 61/657,794is incorporated herein by reference.

BACKGROUND

Digital graphic design and media editing applications (hereaftercollectively referred to as image editing applications or media editingapplications) provide graphical designers, media artists, and otherusers with the necessary tools to view and edit an image. Examples ofsuch applications include Aperture®, Final Cut Pro®, iMovie®, andiPhoto®, all sold by Apple, Inc. These applications give users theability to edit images in a variety of manners. For example, someapplications provide different range sliders for adjusting differentproperties of an image or a video.

Many media editing applications, however, do not provide automatic imageadjustment controls. For example, a user is required to manually adjustdifferent sliders or enter values for different image properties inorder to effectively use most of the existing image adjustment tools.This requires the user to have extensive knowledge of the values andslider positions for making different image adjustments. Unless the userknows precise values or slider positions for making image adjustments,the user is forced to try different values or slider positions until theadjustments are accurate. Even a user with such extensive knowledge maynot have precise values or slider positions readily available when theuser is making image adjustments. Furthermore, the user is left toverifying the accuracy of the adjustment by simply eye-balling the imageafter the adjustment. These deficiencies cause unnecessary inconveniencein editing an image.

BRIEF SUMMARY

A media editing application of some embodiments performs a novelautomatic exposure adjustment of an image. The automatic exposureadjustment in some embodiments multiplies color component or luminancevalues (e.g., RGB or one or more values in a YCC color space) of pixelsin the image by some multiplier value to produce new image exposurevalues. In some embodiments, the new values are capped at a maximumcolor component or luminance value. The application uses a method todetermine the multiplier value used to multiply the color componentvalues. In some embodiments, the method computes a histogram based on aset of luminance values of the image and identifies a distance to aparticular point in the histogram in order to identify the multipliervalue for adjusting the luminance of the image.

The method of some embodiments identifies a multiplier value that willdarken the image based on a distance to black in the histogram. Adistance to black in some embodiments is a distance from the darkestpossible black to some low percentile of pixel luminance (among thedarker pixels) in the image. Similarly, some embodiments identify amultiplier value that will brighten the image based on a distance towhite in the histogram. A distance to white in some embodiments is adistance from the brightest possible white to some high percentile ofpixel luminance (among the brighter pixels) in the image.

Some embodiments provide a device that stores a media editingapplication having a graphical user interface (GUI) for automaticallyadjusting a set of image parameters of an image displayed in a previewdisplay area of the GUI and modifying the appearance of the displayedimage based on the set of adjusted parameters. In some embodiments, themedia editing application automatically adjusts the set of imageparameters in response to a user input to adjust the set of parametersfor the displayed image.

Some embodiments adjust exposure of an image based on a temperedexposure value identified based on an input exposure value of thereceived image and a target exposure value. The target exposure value ispre-determined in some embodiments.

Some embodiments provide a method for tempering an adjustment of animage to account for prior adjustments to the image. The adjustment insome embodiments is an automatic exposure adjustment. In someembodiments, the method adjusts an image received by a media editingapplication. The method compares a set of existing parameters to adifferent set of parameters. Based on the comparison, the methodperforms an operation to adjust the image.

In some embodiments, the method performs a set of post-processingoperations for a first set of adjustments on a first set of parameters(e.g., color saturation, image sharpness, average luminance) of theimage received by the media editing application. The first set ofparameters of the received image quantify a set of prior adjustments tothe image before the image is received by the media editing application.The set of prior adjustments are adjustments that were performed by animage capturing device when the image was captured in some embodiments.

The method compares the first set of parameters to a second set ofparameters to produce a third set of parameters that expresses thedifference between the first set of adjustments and a second set ofadjustments. The second set of parameters is a set of target or desiredparameters in some embodiments. The method performs a third operationbased on the third set of parameters to produce an adjusted image. Insome embodiments, the third set of parameters are between the first andsecond sets of parameters, allowing the third operation to temper theimage adjustment by moderately adjusting the first set of existingparameters toward the second set of desired parameters.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description, and Drawings is needed. Moreover,the claimed subject matters are not to be limited by the illustrativedetails in the Summary, Detailed Description, and Drawings, but ratherare to be defined by the appended claims, because the claimed subjectmatters can be embodied in other specific forms without departing fromthe spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 conceptually illustrates a process of some embodiments thatperforms an automatic image adjustment operation that tempers theadjustment to account for prior adjustments to the image.

FIG. 2 illustrates a process pipeline that makes a fractional adjustmentto a computed Delta value to adjust the exposure of a captured image insome embodiments of the invention.

FIG. 3 conceptually illustrates a slider control for modifyingparameters of an image of some embodiments.

FIG. 4 conceptually illustrates an auto exposure adjustment module thatautomatically identifies an exposure value of an image using a functionthat computes a tempered target intensity for the image.

FIG. 5 illustrates two examples of computed tempered intensity valuesfor two input images that each has an average intensity value that isless than the target intensity value.

FIG. 6 illustrates two other examples of computed tempered intensityvalues for two other input images that each has an average intensityvalue that is greater than the target intensity value.

FIG. 7 conceptually illustrates three different tempering functions ofsome embodiments that are each selectable to temper a target averageluminance based on an analysis of the image's histogram.

FIG. 8 conceptually illustrates a process of some embodiments thatadjusts the exposure of an image when an average luminance of the imageis not the same as a target luminance for the image.

FIG. 9 illustrates different image histograms having the same averagecomputed luminance but different tempering curves for decreasing imageluminance based on different distances to black of some embodiments.

FIG. 10 illustrates an example of a GUI for adjusting image luminancewhen the tempering curve is based on a short distance to black.

FIG. 11 illustrates another example of a GUI for adjusting imageluminance when the tempering curve is based on a long distance to black.

FIG. 12 illustrates different image histograms having the same averagecomputed luminance but different tempering curves for increasing imageluminance based on different distances to white of some embodiments.

FIG. 13 illustrates an example of a GUI for adjusting the luminance ofan image when the tempering curve is based on a short distance to white.

FIG. 14 illustrates an example of a GUI for adjusting the luminance ofan image when the tempering curve is based on a long distance to white.

FIG. 15 provides two examples that illustrate the difference between afull exposure adjustment of an original image and a tempered exposureadjustment of the image.

FIG. 16 provides two more examples that illustrate the differencebetween a full exposure adjustment of the original image and a temperedexposure adjustment of the image.

FIG. 17 conceptually illustrates an example of an electronic system withwhich some embodiments are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

Some embodiments of the invention provide an image-editing process thatperforms an auto-adjustment operation that automatically adjusts animage but tempers its adjustment to account for prior adjustments to theimage. FIG. 1 conceptually illustrates this process for some embodimentsof the invention. The process 100 of this figure is performed by a mediaediting application in some embodiments. In some embodiments, this mediaediting application executes on top of the operating system of a device(e.g., a computer, a camera, a smartphone, etc.), while in otherembodiments it is part of the device's operating system.

The process 100 of some embodiments begins by receiving (at 110) animage that has been adjusted by a first image processing operation. Insome embodiments, the received image has been processed prior to themedia editing application receiving the image. For example, the imagemay have been originally generated by an image-capturing device (i.e., acamera) that processed the image based on data that it captured from ascene (i.e., the light captured from the scene).

Next, the process 100 identifies (at 120) a first set of parameters thatquantifies a first set of adjustments to the image based on the firstimage processing operation. In some embodiments, the image has beenprocessed multiple times prior to the media editing applicationreceiving the image (e.g., during image capture and then during animport process into the media editing application). In this example,therefore, any processing of the image prior to the media editingapplication receiving the image may be treated as the first imageprocessing operation, and the first set of adjustments are thoseadjustments made to the image as received. However, in some embodiments,the first image processing operation is an image adjustment operationthat an image capturing device (e.g., a camera) performs when itcaptures the image, and this operation is based on data that the cameradetected in the scene as it was capturing the image. In someembodiments, the first set of adjustments is a set of image adjustments(e.g., color processing adjustments, white balance adjustments,brightness adjustments, other tonal adjustments, etc.).

After the first set of adjustments is quantified, the process 100identifies (at 130) a second set of parameters that quantify a desiredsecond set of adjustments to the image based on a second imageprocessing operation. To obtain the second set of parameters, theprocess 100 performs the second image processing operation on the imagein some embodiments. In other embodiments, however, the process 100identifies the second set of parameters without performing the secondimage processing operation on the image. For instance, in someembodiments, the second set of parameters is a set of target parameterstoward which the first set of parameters are to be adjusted. The set oftarget parameters includes pre-specified parameter(s), derivedparameter(s) computed for the image, or both.

Next, the process 100 compares (at 140) the first and second sets ofparameters to produce a third set of parameters that expresses thedifference between the first and second sets of parameters. Based on thethird set of parameters produced by comparing the first and second setsof parameters, the process 100 performs (at 150) a third imageprocessing operation on the image to produce an adjusted image. Afterproducing the adjusted image by performing the third image processingoperation, the process 100 ends.

In some embodiments, the process 100 is used to adjust images that havebeen previously adjusted by an image-capturing device (e.g., a camera)at the time that the device captured the image. Specifically,image-capturing devices estimate exposure for all non-manual photostaken to automatically expose photos “correctly”. Typically, such adevice takes into account many factors related to the sensor, theenvironment, and the scene to determine settings for one or more of thefollowing to achieve correct exposure: ISO, shutter speed, aperture,flash. However, some photos are still under or over exposed, sometimesdue to unusual scenes, sometimes as a result of sensor limitations, andsometimes because of user error.

For such cases, the process 100 can adjust an image's attributes (e.g.,exposure, white-point, focus, etc.) either as a post-processingoperation that is performed after exporting the image from theimage-capturing device, or as a secondary operation that theimage-capturing device performs on-board. The process's image editingoperation in some embodiments can be characterized as follows. In thisexplanation, Primary refers to the image processing operation performedby the device, while Secondary refers to the subsequent, tempered imageprocessing operation performed by the process 100. The tempered imageprocessing operation is tempered from an operation that would adjust theimage to meet a pre-set target (e.g., a target average luminance of thepixels). In some embodiments, the process 100 computes a Delta, whichexpresses the difference between the effect of the target adjustment andthe effect of the Primary (e.g., Delta=Target_Effect−Primary_Effect). Toexpress these effects, the process 100 identifies a set of attributes inthe images that result from the Primary and Target operations. In thecase of exposure, the set of attributes includes the average luminancevalue in some embodiments, as further described below. After computingthe Delta, the process computes a fraction (between 0 and 1) of theDelta based on a set of criteria that provides a measure of confidencein the effect of the Target operation with respect to the effect of thePrimary operation. The fraction is computed, in some embodiments, tomoderate the effect of the Target operation to produce the secondaryoperation. For instance, a small Delta value may indicate a considerableamount of confidence that performing the Target operation on the image'sattributes will correctly adjust the image's attributes. In contrast, alarge Delta value may indicate only a slight amount of confidence thatperforming the Target operation on the image will correctly adjust theattributes. Thus, the fraction can be computed so that of the effect ofthe Secondary operation is mitigated either to a greater extent or to alesser extent based on the measure of confidence indicated by the Deltavalue. After computing the fraction, the process selects the set ofparameters for the Secondary operation to achieve this fractionaladjustment to the image. In some embodiments, the process selects theset of parameters for the Secondary operation to achieve the fractionaladjustment based on an assumption that the adjustment (e.g., exposurecorrection) performed by the image-capturing device was useful since ithad several data points about the scene at the time the image wascaptured. In the example of exposure correction, the set of parametersfor the Secondary operation may include the computed Delta and apre-defined target luminance value, as further described below inSection III.

FIG. 2 illustrates how the process 100 makes a fractional adjustment toa computed Delta value to adjust the exposure of a captured image insome embodiments of the invention. The image processing pipeline in thisfigure includes an image analyzer 210, a delta identifier 220, anexposure value calculator 230, and an exposure adjustor 240. The imageanalyzer 210 receives an image 205 from a storage 250. It computes a setof parameters (e.g., an average luminance value) that quantifies a setof characteristics of the image.

The image analyzer 210 then provides this computed set of parameters tothe delta identifier 220. The delta identifier 220 also receives atarget set of parameters (e.g., a target average luminance value) fromthe storage. The delta identifier 220 then computes a fractional deltavalue based on the difference between the computed set of parameters andthe received target set of parameters.

The delta identifier 220 then supplies the fractional delta value to theexposure value calculator 230. The exposure value calculator 230 thencomputes an exposure value based on the received delta value. Forinstance, as further described below, the exposure value calculator 230receives a fractional delta value that expresses a tempered, targetaverage luminance value in some embodiments. From this received value,the exposure value calculator computes an exposure value and passes thecomputed exposure value to the exposure adjustor 240. In someembodiments, the computed exposure value is used to set a control (e.g.,a slider control) for adjusting the exposure of the image. The exposureadjustor 240 uses the computed exposure value to adjust the exposure ofthe retrieved image to achieve the desired tempered target value (e.g.,the desired, tempered average luminance value).

Several more detailed embodiments are described below. Section Idescribes an implementation of an exposure slider control for adjustingexposure of images in a media editing application. Section II describesin an implementation of an exposure slider control that automaticallyadjusts image intensity based on delta computations. Next, Section IIIdescribes several examples of using different curves based on differentdistances to black and/or white to temper the luminance adjustments toan image. Described in Section IV is the software architecture of themedia editing application that uses the exposure slider control of someembodiments. Lastly, Section V describes a computer system thatimplements some embodiments of the invention.

I. Exposure Slider Control

The exposure of an image depends on many factors. For instance, an imagecapturing device that captures an image of a scene may estimate exposurebased on image parameters such as ISO, shutter speed, aperture, andflash. However, due to various circumstances (e.g., an unusual scene,camera sensor limitations, user error, etc.) the exposure estimation mayresult in an image that is underexposed or overexposed. In these cases,it may be desirable to adjust exposure. In some embodiments, theexposure of an image can be modified by adjusting one or more parametersof the image.

FIG. 3 conceptually illustrates a slider control 370 for modifying oneor more parameters of an image. A user moves this slider control 370 toselect different multiplier values to modify the parameters of the imagedifferently. FIG. 3 illustrates the operation of the slider control 370in terms of three stages (310-330) in which a user moves a slider knob380 to different positions along a slider track 390 that are associatedwith different multiplier values. At each stage in this figure,different response graphs 340, 350, and 360 are shown alongsidehistograms 347, 357, and 367, respectively.

Each histogram represents a set of tonal values of an image. In someembodiments, the histogram represents the image by plotting each pixelof the image to a tonal value. For instance, an image having a set ofpixels may have a first sub-set of pixels with a first tonal value, asecond sub-set of pixels with a second tonal value, and a third sub-setof pixels with a third tonal value. The histogram may chart first,second, and third positions along the x-axis for the first, second, andthird tonal values, respectively. Then, along the y-axis, the histogrammay illustrate the frequency or number of pixels at each tonal value.Based on the plotted pixel values, a curve is formed (or conceptualized)that illustrates the distribution of tonal values for the image. Forexample, at each of the three x-axis positions, a line or a bar that maybe displayed along the y-axis of the histogram to represent the number(or frequency) of pixels having the corresponding tonal value along thex-axis.

In some embodiments, the histograms are brightness histograms for animage. A brightness histogram is a histogram of a brightness attribute,such as luminance or luma component color value, of the image. Thebrightness histogram of the image represents an approximation of thebrightness of a scene captured or received by an image capturing device(e.g., a camera). Such a brightness histogram illustrates a range ofdifferent image pixel values having different amounts of brightness. Forinstance, different image capturing devices can approximate the range ofbrightness of the captured scene differently. In some embodiments, thebrightness histogram spans the full dynamic range of all possible tonalvalues for the image. Within the full dynamic range, an exposure rangeof the image is represented by the set of brightness values along thex-axis having one or more plotted image pixels. As shown in FIG. 3, eachof the histograms 347, 357, and 367 represents the dynamic range and theexposure range of brightness for the set of tonal values for the image.

In order to plot the image's pixels on the histogram, in someembodiments, tonal values of the pixels are determined by performingcalculations based on one or more pixel component color values. In someembodiments, an RGB sum is performed to determine the histogram valuefor a particular pixel. In other words, the brightness histogram isexpressed in terms of the sum of the RGB values. In other embodiments,the pixel values are received in the RGB space, converted to a colorspace that has luminance or luma as one of the component color channels(e.g., YCbCr, YUV, YIQ, etc.), and the brightness histogram isconstructed from the luminance or luma values. Yet other embodimentsconstruct other types of histograms to express the range of brightnessand tonal variations of the image.

In some embodiments, an overall intensity of the image is determinedbased on the tonal values of the image's pixels. In some embodiments,the overall intensity of the image is based on the average brightness ofthe pixel values of the image. In other embodiments, the overallintensity of the image is based on the brightness value of the medianpixel in the distribution of pixels of the histogram. The median pixelin the distribution of pixels of the histogram is the median pixel ofall the pixels of the image sequentially ordered. In some embodiments,an exposure value (EV) represents the overall intensity of the image.The EV can be indicated by a number of “stops” or “f stops”. The overallintensity of the image in some of these embodiments is modified based onchanges to the EV. In some of these embodiments, the EV for a number ofstops (N) is determined from a base-2 logarithmic scale or calculation,as shown in the equation:EV=log₂(N)  (1)

In these embodiments, linear changes to the EV (+1 stop, +2 stops, −1stop, −2 stops, etc.) result in exponential changes to the overallbrightness of the image. For instance, increasing the exposure of theimage by one stop (EV+1) doubles the average brightness of the image(e.g., the brightness value of each image pixel is multiplied by two).Similarly, decreasing the exposure by a single stop (EV−1) halves theaverage brightness of the image (e.g., the brightness value of eachimage pixel is divided by two).

The exposure slider control 370, in some embodiments, associates a setof positions along the slider track 390 to a set values (e.g., a set off stop values) for modifying the exposure parameters (e.g., luminancevalues) of an image. The positions of the slider control 370 areassociated, in some embodiments, with pre-determined values along an EVscale (e.g., −3, −2, . . . , 3). An initial setting of the slider knob380 for an original image is at a default position (e.g., zero position)along the slider track 390 that is associated with no change (e.g.,EV0). The other slider control 370 positions represent relative changesin the EV of an image. In some embodiments, when the user moves theslider knob 380 to the left, the original image exposure parameters(e.g., luminance values) are modified (e.g., reduced) based on themultiplier associated with the position selected along the track 390 ofthe slider control 370. Likewise, when the user moves the slider knob380 to the right, the original image parameters (e.g., luminance values)are modified (e.g., increased) based on the multiplier value that isassociated with the user selected position on the track of the slidercontrol 370.

In some embodiments, slider knob 380 movements between differentpositions are dynamically captured along the slider track 390 formodifying the exposure parameters (e.g., luminance values) betweenstatic EV positions associated with pre-determined values. For example,movements of the slider knob 380 along the portion of the slider track390 between EV0 and EV+1 may be identified to change the exposure of theimage in unison with the movement of the slider knob 380 (e.g., exposuremodifications occur approximately simultaneously with movements of theslider knob). In these embodiments, the multipliers associated with theslider knob 380 positions can be decimal or fractional values (e.g., 2.5or 5/2) that are continuously captured for changing image exposure asthe user moves the slider knob 380 along the slider track 390.

The response graphs, 340, 350, and 360, illustrate the effect of auser's movement of the slider knob 380 along the track of the slidercontrol 370 to select a multiplier value. As shown, each response graph340, 350, and 360 includes a response curve, 345, 355, and 365,respectively. The response curves indicate changes in the parameters ofthe image when a selected multiplier is applied to the parameter values.For example, a set of exposure values that represent original luminancevalues of an image (i.e., luminance values exposure adjustments via theslider control 370) can be modified when a user moves the slider knob380 to a position that is associated with a multiplier value foradjusting the exposure of the image. In FIG. 3, a set of exposure values(e.g., luminance values) is illustrated along the x-axis of eachresponse graph, 340, 350, and 360. Also shown along the y-axis of theresponse graphs, 340, 350, and 360, are different sets of brightnessvalues (e.g., luminance values) that represent the modified brightnessvalues of the images.

Based on the multiplier value selected for adjusting the brightness ofan image, a slope for the adjustment response curve is determined. Theslope represents the magnitude of change for the brightness parametersof the image. Likewise, the slope of the response curve indicates thevalue of the multiplier for modifying the brightness parameters of theimage. In some embodiments, where there is no change to the brightnessparameters of the image, the response curve is a straight line having aslope of one (i.e., a 45° line with respect to the x-axis and y-axis).In these embodiments, all of the input brightness parameter values(e.g., luminance values) are mapped to the same output brightnessparameter values. However, when the image's brightness is changed, theadjustment response curve is reformulated to map the input brightness(e.g., luminance) parameter values to the corresponding outputbrightness parameter values according to the response curve slope. Insome embodiments, the input brightness parameters correspond to theoutput brightness parameters based on the selected multiplier specifiedby the position of the slider knob 380.

The first stage 310 of FIG. 3 shows the slider control 370 with aninitial setting of zero (i.e., the slider knob 380 is at the zeroposition). In some embodiments, the initial setting is not associatedwith any change in the image exposure parameters. The overall image isrepresented at this stage 310 by the histogram 347, which charts thedistribution of pixel values for the image. As there is no change in theposition of the slider knob 380 at this stage 310, the response curve345 is shown at a 45° angle (i.e., slope m=1) in the response graph 340.

The second stage 320 shows that the user moves the slider knob 380 onestop (i.e., EV+1) along the track 390 of the slider control 370. In thisexample, the slider knob 380 position is associated with a multipliervalue of two based on the specified EV+1 change value. Applying theselected multiplier value to the image doubles the image parameters(e.g., twice as much luminance). To perform this doubling operation, insome embodiments, the value of each image pixel in the histogram 347 isdoubled. This results in a higher average pixel luminance value. In someembodiments, when a pixel value is increased to more than a maximumallowed value (e.g., a limiting value of a display device or a limitingvalue of an image format), the pixel's value is capped at the maximumvalue. In other embodiments, the image pixel value is allowed to doublewhen the value is increased beyond the maximum value. However, in theseembodiments, the pixel may be displayed (e.g., on a display screen) atthe maximum exposure value for the image (e.g., the brightest white),while retaining (e.g., in a data structure storing data of the image) alogical value beyond the maximum allowed value.

The histogram 357 illustrates the effect of doubling the pixel luminancevalues. In this case, the doubling of the pixel luminance values causesa substantial shift in the distribution of pixels to the right of thehistogram. As shown, the number of pixels represented on the right sideof the histogram 357 is substantially more than the number of pixels onthe right side of the histogram 347 (i.e., before applying the selectedmultiplier value). In particular, a large spike is shown at the farright (i.e., the white edge) of the histogram 357. This large spikeillustrates the shift in pixel values to the right and the capping ofthe pixel values that exceed the limiting value (e.g., 255). Likewise,the number of pixel values on the left side of the histogram 357 issubstantially less than the number of pixel values on the left side ofthe histogram 347. As the histograms 347 and 357 chart the relativebrightness of the pixel luminance values from left to right (i.e.,darkest to lightest), the shift in pixels shown between these histograms347 and 357 indicates the increase in overall luminance for the image.

The response curve 355 at this stage 320 also illustrates the effect ofdoubling the pixel luminance values. As shown in the response graph 350,the slope of the response curve 355 has increased from one, in theresponse curve 345, to two. Thus, each input value shown along thex-axis of the response graph 350 is mapped to an output value (y-axis)that is twice the input value. Although the y-axis is conceptuallyillustrated as having values to 512, the actual brightness values getcapped, in this example, at a limiting value associated with the image(e.g., limit of 255). In other embodiments, the values beyond themaximum are retained but are limited in the display of the image. Forexample, an 8-bit JPEG image may be limited to values in the range of0-255, while a 12-bit RAW image may support values well beyond 255. Asthe slope of the response curve 355 at this stage 320 is two, the outputvalues of several input pixel values (i.e., input pixel values from 0 to127) are less than the limiting value. As shown in the response curve355, the values are doubled. As each input value in the response curve355 is doubled to the corresponding output value, the overall brightnessof the image would theoretically double. However, the output values ofseveral input pixel values (i.e., input pixel values from 128 to 255)exceed the limiting value. These output values are capped at 255, asshown in the response curve 355. Thus, several pixels having differentimage pixel values before the multiplier value is selected by the slidercontrol 370 are capped at the maximum value. In some embodiments, suchdoubling and capping results in a loss of image detail or information(e.g., relative luminance differences are lost).

In other embodiments, the maximum allowed value only represents amaximum allowed value for displaying the image. In these embodiments,the values would increase, but the visible display of pixels havingvalues that exceed the maximum allowed value would be limited to themaximum. For instance, pixel values of 128 are doubled to 256 and pixelvalues of 255 are doubled to 510. The output values (256 and 510),however, exceed the maximum allowed value for displaying the image inthis example. Thus, several pixels having different values before themultiplier value is selected by the slider control 370 are increased tovalues that exceed the maximum allowed value, but the visible display ofthose pixels is limited to the maximum allowed value. In theseembodiments, the display of the capped pixels could result in a loss ofvisible image detail or information, even though the pixel values beyondthe maximum allowed value are retained.

The third stage 330 shows that the user moves the slider knob 380 to theleft to select a multiplier value of ½ based on an EV−1 change specifiedby the new position of the slider knob 380. The multiplier selected inthis case is a fraction, which when applied to image pixel values,causes reductive changes in value (e.g., reduces luminance values tohalf their original values). To perform this exposure reductionoperation, in some embodiments, the value of each image pixel in thehistogram 347 is halved (resulting in histogram 367). As shown in theresponse graph 360, the slope of the response curve 365 has decreasedfrom one (i.e., in the slope of the response curve 345) to ½ based onthe selected EV−1 value associated with the multiplier value of ½. Thus,when the selected multiplier value (½) is applied to the image, theluminance values are reduced by half.

To perform this reduction operation, which results in a lower averageluminance value for the image, the luminance value of each pixel in thehistogram 347 is halved. The histogram 367 illustrates the effect ofhalving the pixel luminance values. In this case, the halving of theimage pixel values has caused a substantial shift in the distribution ofpixels to the left side of the histogram. As shown, the number of pixelsrepresented on the left side of the histogram 367 is substantially morethan the number of pixels represented on the left side of the histogram347 (i.e., before applying the multiplier). Furthermore, all of thepixel luminance values on the right side of the histogram 347 haveshifted to the left side in histogram 367, so that no pixels arerepresented on the right half of the histogram 367. As the vast majorityof pixels have been redistributed from the right (e.g., brighterluminance) to the left (e.g., darker luminance) of the histogram, theaverage image pixel value (e.g., overall exposure) has decreased.

Also, the response curve 360 illustrates the effect of reducing thepixel luminance values by half. In particular, each input value, asshown along the x-axis of the response graph 360, is mapped to an outputvalue that is half the input value at this stage 330. As shown in thisexample, pixel luminance values of 64 are halved to 32, pixel luminancevalues of 96 are halved to 48, pixel luminance values of 128 are halvedto 64, pixel luminance values of 160 are halved to 80, and pixelluminance values of 208 are halved to 104. Furthermore, none of theinput pixel luminance values are output to luminance values of 128 ormore.

In the examples described above, and further described below, the datacalculations are shown as whole number (i.e., integer) calculations.However, in some embodiments, some or all data calculations andcomputations are made with floating point values. In these cases, theimage adjustments are based on floating point computations that are moreprecise (e.g., no round-off loss of data within a series of cumulativecalculations) than computations based on integer values. In some cases,visual image detail is preserved by treating all values (i.e., integerand decimal values alike) as floating point values in order to performany calculations or computations for making image adjustments.

While the user's selection of a slider position (e.g., f stop value)along the slider track 390 of the slider control 370, as described forFIG. 3, causes the exposure intensity of the image to be modifiedaccording to the multiplier value associated with the selected sliderposition, in other cases, exposure values are automatically identified.In some embodiments, an exposure enhancement process uses a function toautomatically identify the exposure value. FIG. 4, which is described byreference to FIGS. 5 and 6, conceptually illustrates an auto exposureadjustment module that performs such an exposure enhancement process byusing a function to automatically identify the exposure value.

II. Delta Computation for Auto Intensity Adjustments

In some embodiments, image intensity is quantified according to tonalvalues of the image's pixels. For example, the average computedbrightness of the image can be identified, in some cases, by the meanbrightness value of all the pixel brightness values of the image. Inother cases, the average computed brightness can be identified by themedian brightness value in a histogram for the image. In some cases,however, we want to change the intensity (e.g., average computedbrightness or luminance) to a target intensity. The target intensity canbe different intensity values in different embodiments (e.g., a 50%brightness value, 18% brightness value, etc.). As the identifiedintensity of an image may be determined to be more or less intense thanthe desired level of intensity for the image, adjustments are needed tomatch the target intensity. However, in some cases, adjusting theidentified intensity to match the target intensity could result inunintended image distortion or other consequences. For example, a targetintensity that cranks up the brightness of the image in order tocorrectly expose a set of under-exposed image elements (e.g., faces ofpeople in a background portion of the image) may inadvertentlyover-expose another set of well-exposed image elements (e.g., faces ofpeople near the image foreground). Thus, in some embodiments, thedifference (also known as the “delta”) between the current intensityvalue for the image and the desired intensity is determined. Intensityadjustments are then tempered or moderated based on the calculateddelta, in order to adjust the intensity to some level between a fulladjustment (i.e., adjusting intensity all the way to the targetintensity) and no change to the intensity (i.e., retaining the inputintensity of the image without any adjustment). The following sectiondescribes such an approach that compromises between a full adjustmentand no adjustment. In some embodiments, a tempered adjustment isidentified by using a function which, based on the calculated delta,automatically computes an exposure value that is associated with amultiplier value for adjusting image intensity.

FIG. 4 conceptually illustrates an auto exposure adjustment module 410that automatically identifies the exposure value using a function thatcomputes a tempered target intensity based on the set of input imageparameters and a target intensity for the image. In some embodiments,the auto exposure adjustment module 410 is a user-selectable tool whichwhen selected identifies an exposure value 435 for performing an autoadjustment operation on the received image. The auto exposure adjustmentmodule 410 identifies the exposure value 435 by computing a temperedintensity value based on a computed intensity parameter of the receivedimage and a target intensity value for the image. The operation of theauto exposure adjustment module 410 is described below by reference toFIGS. 5 and 6, which conceptually illustrate four examples of computedtempered intensity values for four different images.

As shown in FIG. 4, the auto exposure adjustment tool 410 includes animage analyzer 420, an exposure value identifier 430, and an exposureadjustor 440. In some embodiments, the image analyzer 420 analyzes oneor more parameters of an original image 455 to identify the intensityvalue of the image.

In some embodiments, the computed intensity parameter is a luminancevalue that expresses an average luminance for a set of pixels of theoriginal image 455. Different embodiments compute the average intensityparameter differently. Examples of such computed average intensityparameters include median, mean, average, or any other measure ofcentral tendency in a distribution of values.

As shown in FIG. 4, the image analyzer 420 retrieves the image toanalyze from an image storage 450. The image storage 450, in someembodiments, is a storage of an image editing application. In some ofthese embodiments, images that are imported into the image editingapplication are stored in the image storage 450. In some embodiments,the image storage 450 is a storage of a device (e.g., a digital camera)that is used to capture the image.

As further shown in FIG. 4, the image analyzer 420 provides the averageintensity value that it computed for the retrieved image to the exposurevalue identifier 430. The exposure value identifier 430, in someembodiments, identifies the exposure value 435 by computing a temperedintensity value based on the average intensity value and the targetintensity value.

In some embodiments, the tempered intensity value is a solution formodifying the average intensity value of the original image 455 togenerate a tempered target intensity (T′ as shown in the breakout box460) for the image. For example, the tempered intensity value is asolution for modifying the original average intensity value that isshort of matching the target intensity value. In some embodiments, thetempered intensity value is calculated to optimize the average intensityvalue for the image. In some of these embodiments, the exposure valueidentifier 430 identifies a tempered target intensity (T′ as shown inthe breakout box 460) and then computes the tempered intensity value(TV, as shown in the breakout box 460) based on the tempered targetintensity. In some other embodiments, the exposure value identifier 430identifies the tempered target intensity (T′) and then retrieves thetempered intensity value from a storage (e.g., a lookup table, adatabase, etc.) based on the tempered target intensity.

The exposure value identifier 430, as shown in FIG. 4, also provides theidentified exposure value 435 to the exposure adjustor 440. In someembodiments, the exposure adjustor 440 receives the exposure value 435from the exposure value identifier 430 in order to adjust the exposureintensity of the original image 455 based on the exposure value. In someembodiments, the exposure adjustor 440 retrieves the original image 455from the image storage 450. In some of these embodiments, the exposureadjustor 440 retrieves the original image 455 when it receives theexposure value 435 from the exposure value identifier 430. In some otherembodiments, the exposure adjustor 440 retrieves the original image 455at approximately the same time as when the image analyzer 420 retrievesthe original image 455. Based on the exposure value, the exposureadjustor 440 adjusts the exposure intensity of the image to generate aprocessed image 465. In some embodiments, the exposure adjustor 440outputs the processed image 465 to one or more other modules.

Having described the individual components of the auto exposureadjustment module 410, the operation of FIG. 4 is described by referenceto four examples illustrated in FIGS. 5 and 6. FIG. 5 illustrates twoexamples of computed tempered intensity values for two input images thateach has an average intensity value that is less than the targetintensity value. A first input image having an average intensity valueof 0.45 is used to compute the tempered intensity value in a firstexample of FIG. 5. A second image having an average intensity value of0.25 is used to compute the tempered intensity value in a second exampleof FIG. 5.

As shown in different stages of FIG. 5, an intensity indicator 570conceptually illustrates a value of the intensity currently computed foran image. Unlike the slider control 370, which demonstrates relativeexposure value adjustments (e.g., EV+1 doubles the intensity of theimage, while EV−1 halves the intensity of the image) for each positionon the slider track 390, the intensity indicator 570 demonstrates thecurrently computed value of the image's intensity at the position of theslider 545.

Also, the response graphs 530 and 550 shown in FIG. 5 illustrate a stateof no change for an image that will be processed. These response graphs530 and 550 are shown to conceptually illustrate that the exampledescribed for FIG. 5 includes a target intensity (i.e., the horizontaldashed line at y-axis value 0.5 in these examples) and a currentintensity (i.e., the darkened dashed line at 0.45 of the x-axis shown inthe first example). In other words, the response graphs 530 and 550 donot illustrate a change of any image values, but instead provide contextfor describing subsequent computations that illustrate the compromisebetween making a full intensity adjustment to the target intensity andmaking no intensity adjustment. Furthermore, unlike the valuesillustrated along the x-axis and y-axis of the response graphs in FIG.3, the values along the x-axis of the response graphs in FIG. 5 are notmapped to the values along the y-axis. In this example, the valuesillustrate the result of making tempered adjustments to the intensity ofthe image. Therefore, the individual pixel values of the image are notshown in the response graphs as being mapped to a corresponding outputvalue. Instead, for example, a value that is shown along the x-axis ofthe response graph can be an average intensity value of all the pixels,while the corresponding value along the y-axis can be the averageintensity value of all the pixels after making a tempered intensityadjustment (i.e., in response graphs 540 and 560).

In a first stage 510 of the first example, the response graph 530 isshown which represents intensity values (e.g., average computedluminance) before and after an intensity adjustment to the originalimage 455 that the image analyzer 420 retrieves from the image storage450. In this example, a straight 45° response curve 535 is shown. Inparticular, the intensity value of 0.45 along the x-axis corresponds tothe intensity value of 0.45 along the y-axis. This is shown by the bolddashed lines connecting at the response curve 535. As there is no changein the image intensity shown in this response graph 530, the valuesalong the x-axis and y-axis are the same at this stage 510.

The target intensity value of 0.5 is also shown by the faint horizontaldashed line across the response graph 530. The target intensity, in someembodiments, is a pre-determined value of a desired intensity for anyimage. With respect to FIG. 4, the image analyzer 420 identifies theinitial intensity value for the image based on the average computedintensity of the image. As shown at this stage 510, the slider 545 ispositioned slightly to the left of the 0.5 mark on the intensityindicator 570. The image analyzer 420 then provides the averageintensity value of the image to the exposure value identifier 430.

In a second stage 520 of the first example, the response graph 540 isshown with a curve 537 for adjusting the intensity value (e.g., averagecomputed luminance) of the image. The curve 537 is based on the firstfunction (i.e., f_((I, T))) shown in the breakout box 460 of FIG. 4. Insome embodiments, when the difference between the input intensity valueand the target intensity value is sufficiently small, the curve 537closely correlates the input intensity value (e.g., average computedluminance) to the target intensity value. In this example, the inputintensity value (i.e., 0.45) and the target intensity value (i.e., 0.5)differ by a small amount. As shown, at this stage 520, the curve 537closely correlates the input value to an output value at the temperedtarget intensity value of 0.49. In other words, the first function isused to temper the adjustment of the intensity, so that the resultingtempered intensity value (i.e., 0.49) is very close to, but notcompletely at, the target intensity (i.e., 0.5).

Based on the tempered target intensity value (i.e., 0.49), the exposurevalue identifier 430 identifies the new exposure value. In someembodiments, the exposure value identifier 430 determines the newexposure value by using a second function (i.e., G_((I, T′))) as shownin the breakout box 460 of FIG. 4) based on the tempered targetintensity value (T′) and the average intensity value (I) of the image.For example, the exposure value can be determined for a positive changein EV when there is an increase in intensity of the image. The exposurevalue identifier 430 then provides the identified exposure value 435 tothe exposure adjustor 440. Based on the exposure value, the exposureadjustor 440 then identifies the multiplier value to use for adjustingthe intensity value of each pixel in order to modify the averageintensity of the image to the tempered target intensity (i.e., 0.49)calculated in this example. As shown at this stage 520, the slider 545is positioned closer to the 0.5 mark on the intensity indicator 570 toreflect the modified average intensity of the image. In this example,because the delta between the input and target intensities is small, theresulting adjustment toward the target intensity is dramatic—amountingto 80% of the difference between the input intensity value and thetarget intensity value.

In a first stage 550 of the second example, the input intensity value(e.g., average computed luminance) of the original image 455 is 0.25 andcorresponds to the output intensity value of 0.25. Thus, the imageanalyzer 420 identifies the initial average intensity value for theimage, as shown by the slider 545 positioned far to the left of the 0.5mark on the intensity indicator 570. The image analyzer 420 thenprovides this average intensity value (i.e., 0.25) to the exposure valueidentifier 430.

In a second stage 560 of the second example, the curve 537, which isbased on the first function (i.e., f_((I, T))), is shown as correlatingthe input intensity value of 0.25 to an output value of 0.27. Unlike inthe first example, where the difference between input and targetintensity values is sufficiently small, here in the second example,where the difference between the input and target intensity values issufficiently large, the curve 537 significantly mitigates the change inthe intensity. Specifically, the curve 537 correlates the intensityvalue along the x-axis to an intensity value along the y-axis that isfar from the target intensity value. In this example, the inputintensity value (i.e., 0.25) and the target intensity value (i.e., 0.5)differ by a large amount. Accordingly, the input value correlates to avalue that is closer to the target than the input value itself, but thatis substantially closer to input intensity value than it is to thetarget intensity value (i.e., increasing intensity from 0.25 to only0.27).

Based on the tempered target intensity value, the exposure valueidentifier 430 identifies the new exposure value. In some embodiments,the exposure value identifier 430 determines the new exposure valuebased on the tempered target intensity value used in the second function(i.e., G_((I, T′))). The exposure value identifier 430 then provides theidentified exposure value 435 to the exposure adjustor 440. Based on theexposure value, the exposure adjustor 440 then identifies the multipliervalue to use for adjusting the intensity value of each pixel in order tomodify the average intensity of the image (i.e., 0.25) to the temperedtarget intensity (i.e., 0.27). As shown at this stage 560, the slider545 is re-positioned only marginally closer to the 0.5 mark on theintensity indicator 570 to reflect the heavily tempered averageintensity of the image. In contrast to the first example of FIG. 5, theresulting adjustment toward the target intensity, in this example, ismerely 8% of the difference between the input intensity value and thetarget intensity value.

As shown in the first and second examples of FIG. 5, when the averageintensity value is less than the target intensity value, the autoexposure adjustment module 410 modifies all of the pixel intensityvalues to increase the average intensity value toward the targetintensity value. In some embodiments, the amount by which the averageintensity value of the image is increased, depends on the difference(delta) between the average intensity value and the target intensityvalue (e.g., relatively large increases when the delta is small andrelatively small increases when the delta is large). In contrast to theexamples illustrated in FIG. 5, in some cases the average intensityvalue of an image is greater than the target intensity value. In thesecases, the auto exposure adjustment module analyzes the image intensityvalues to identify a slider position associated with a multiplier thatdecreases the average intensity value of the image.

FIG. 6 illustrates two examples (i.e., third and fourth examples inreference to the description of FIG. 4) of computed tempered intensityvalues for two input images that each has an average intensity valuethat is greater than the target intensity value. A first input imagehaving an average intensity value of 0.55 is used to compute thetempered intensity value in the third example. A second image having anaverage intensity value of 0.75 is used to compute the temperedintensity value in the fourth example.

In a first stage 610 of the third example, the response graph shows thatthe input intensity value (e.g., average computed luminance) of theoriginal image 455 is 0.55 and is correlated to the output intensityvalue of 0.55. Thus, the image analyzer 420 identifies the initialintensity value for the image, as shown by the slider 545 positionedslightly to the right of the 0.5 mark on the intensity indicator 570.The image analyzer 420 provides this average intensity value (i.e.,0.55) to the exposure value identifier 430.

In a second stage 620 of the third example, the response curve 637,which is based on the first function (i.e., f_((I, T))), correlates theinput intensity value of 0.55 to an output value of 0.51. Unlike in thefirst example (i.e., at stage 520 of FIG. 5), in which the inputintensity value is increased, the input intensity value shown in thisexample is decreased. The amount of decrease depends on the functionused to temper the intensity adjustment and the difference between thetarget intensity value and the input intensity value. Specifically, whenthe delta between the input intensity value and the target intensityvalue is sufficiently small, the curve 637 closely correlates the inputintensity value (e.g., average computed luminance) to the targetintensity value. Thus, the input intensity value (i.e., 0.55) isdecreased to a tempered target intensity value (i.e., 0.51) that isclose to the target intensity value (i.e., 0.5).

Based on the tempered target intensity value, the exposure valueidentifier 430 identifies the new exposure value. In some embodiments,the exposure value identifier 430 determines the new exposure valuebased on the tempered target intensity value used in the second function(i.e., G_((I, T′))). The exposure value identifier 430 then provides theidentified exposure value 435 to the exposure adjustor 440. Based on theexposure value, the exposure adjustor 440 then identifies the multipliervalue to use for adjusting the intensity value of each pixel in order tomodify the average intensity of the image (i.e., 0.25) to the temperedtarget intensity (i.e., 0.27). As shown at this stage 560, the slider545 is re-positioned only marginally closer to the 0.5 mark on theintensity indicator 570 to reflect the heavily tempered averageintensity of the image. In contrast to the first example of FIG. 5, theresulting adjustment toward the target intensity, in this example, ismerely 8% of the difference between the input intensity value and thetarget intensity value.

Based on the tempered target intensity value, the exposure valueidentifier 430 identifies the new exposure value. In some embodiments,the exposure value identifier 430 determines the new exposure valuebased on the tempered target intensity value used in the second function(i.e., G_((I, T′))). The exposure value identifier 430 then provides theidentified exposure value 435 to the exposure adjustor 440. Based on theexposure value, the exposure adjustor 440 then identifies the multipliervalue to use for adjusting the intensity value of each pixel in order tomodify the average intensity of the image (i.e., 0.55) to the temperedtarget intensity (i.e., 0.51). As shown at this stage 620, the slider545 is re-positioned almost all the way to the desired 0.5 targetintensity value on the intensity indicator 570. In terms of the deltacalculation, the tempered adjustment is 80% of the difference.

In a first stage 650 of the fourth example, the input intensity value ofthe original image 455 is 0.75. The difference between the inputintensity value and the target intensity value, at this stage 650, islarge (i.e., 0.25). In the second stage 660 of the fourth example ofFIG. 6, the curve 637 correlates the input intensity value of 0.75 tothe tempered target intensity value of 0.73, which is far from thetarget intensity value of 0.5. Similar to the example shown in FIG. 5,the input intensity and target intensity values differ by a relativelylarge amount, and therefore, the adjustment is heavily tempered based onthe delta. Specifically, the tempered target intensity value after theadjustment is only 8% closer to the target intensity value.

As the third and fourth examples of FIG. 6 show, when the averageintensity value of the image is more than the target intensity value,the auto exposure adjustment module 410 modifies all of the image pixelintensity values to decrease the average intensity value toward thetarget intensity value. In some embodiments, the amount by which theaverage intensity value of the image is decreased depends on thefunction used to adjust the intensity and the delta calculation betweenthe average intensity value and the target intensity value.

III. Different Tempering Curves for Different Distances to Black andWhite

The embodiments described in Section II use one function to temper thetarget intensity and select an exposure value for modifying the exposureof an image. Other embodiments, however, use several functions to temperthe target intensity. Some embodiments select one of several functionsthat computes a tempered target intensity to select an exposure valuefor modifying the exposure of the image. For instance, some embodimentsanalyze the image histogram (e.g., the shape of the histogram orstatistics about the histogram), and based on the analysis, select oneof several functions to use for tempering the target intensity to modifythe exposure of the image. In some embodiments, the selection is made byusing a parameter in a general formula. In these embodiments, differentparameters plugged into the general formula results in the selection ofdifferent functions.

Many of the examples in this application include the selection of atempering curve based on either a distance to black or a distance towhite. In those examples, the curve is used along with an averageluminance value of the image to determine the tempered target averageluminance. However, in some embodiments a particular tempering curve isnot actually calculated. In some embodiments, the distance to black(D_(B)) or the distance to white (D_(W)) and the original averageluminance value are plugged into a multi-variable formula that producesthe tempered target average luminance in a single step. For example,some embodiments use the following formula to generate the temperedtarget luminance value (TL), when the original average luminance value(L) is less than the target average luminance value (T):TL=(T^(1−D _(W)))×((1−L^D _(W)))  (2)

When the original luminance value (L) is more than the target luminancevalue (T), some embodiments use the following formula to generate thetempered target luminance value (TL):TL=1−(((1−T)^(1−D _(B)))×((1−L)^D _(B)))  (3)

In some embodiments, the exposure value (EV) is then determined fromeither of the equations 2 or 3, by the following equation:EV=log₂(TL/L)  (4)

Similarly, some embodiments combine equations 2 and 4 or equations 3 and4 into one large calculation to determine the EV value directly from theoriginal average luminance value, the distance to black (or the distanceto white) value, and the target average luminance value. Some of theseembodiments do not perform intermediate calculations to determine atarget average luminance value as a separate variable.

Furthermore, instead of using the original luminance value (L) tocompute the tempered target average luminance value (TL), someembodiments use the target luminance value (T) and the delta value(i.e., the difference between the target luminance and the originalluminance) to compute the tempered target average luminance value. Thedelta (Δ) is determined in some embodiments based on the followingequation:Δ=T−L  (5)

In these embodiments, the target luminance (T) and the delta (Δ) areplugged into a similar multi-variable formula to produce the temperedtarget average luminance in a single step. For example, some embodimentsuse the following formula to generate the tempered target luminancevalue (TL), when the delta (Δ) is positive:TL=(T^(1−D _(W))×((1−T+Δ)^D _(W)))  (6)

On the other hand, when the delta (Δ) is negative, some embodiments usethe following formula to generate the tempered target luminance value(TL):TL=1−(((1−T)^(1−D _(B)))×((1−T+Δ)^D _(B)))  (7)

In some embodiments, the exposure value (EV) is then determined fromeither of the equations 6 or 7, by the following equation:EV=log₂(TL/(T−Δ))  (8)

Similarly, some embodiments combine equations 6 and 8 or equations 7 and8 into one large calculation to determine the EV value directly from theoriginal average luminance value, the distance to black (or the distanceto white) value, the target average luminance value, and the deltavalue. Some of these embodiments do not perform intermediatecalculations to determine a target average luminance value as a separatevariable.

A. General—Different Curves of Different Functions

FIG. 7 conceptually illustrates three different tempering functions ofsome embodiments that are each selectable to temper the target averageluminance based on an analysis of the image's histogram. Specifically,one of the different tempering functions is selected by theauto-exposure adjustment module 410 based on the distance to black orthe distance to white in the image histogram in some embodiments. Asshown in this figure, the distance to black 750 and the distance towhite 760 are indicated by arrows in each of the histograms A, B, and C(710-730). Also shown in this figure is a response graph 740 and severalcurves 752, 754, 756, 762, 764, and 766, that are based on differentfunctions for tempering the target luminance.

The distance to black 750, in some embodiments, is a measurement fromthe black edge of the histogram (e.g., the darkest possible tonal valueof the image) to a particular percentile of the histogram values. Inother words, a percentage of the total number of image pixels isspecified which encompasses the darkest pixels within the specifiedpercentile (e.g., the 1% of image pixels between the black edge and thefirst percentile). For example, the first percentile (e.g., the darkest1% of pixels) in an image having 6.3 million pixels would encompass the63,000 darkest pixels of the image. The distance to black 750, then, isthe measurement of the distribution in the histogram of those 1% of thedarkest pixels of the image in some embodiments. For example, in ahistogram that plots pixels at 4096 different values (e.g., an image inRAW format), a sparse distribution of the 63,000 darkest pixels of thehistogram may span a portion of the histogram that covers the lowest 10%of histogram values (e.g., 0-409). The distance to black 750 would,therefore, be a relatively large 10% of the histogram. On the otherhand, a dense distribution of the 63,000 darkest pixels may span thelowest 0.1% of histogram values (e.g., 0-3). Here, the distance to black750 would be a relatively small 0.1% of the histogram. While thisexample describes the specified percentage as 1% of the total number ofpixels for the image, other percentages (e.g., 10%, 0.1%, etc.) can beused in other embodiments.

The distance to white 760, in some embodiments, is a measurement fromthe white edge of the histogram (e.g., the brightest possible pixel ofthe image) to a particular percentile of the histogram values. Apercentage of the total number of image pixels is specified which, inthis case, encompasses the brightest pixels within the specifiedpercentile (e.g., the 3% of image pixels plotted in the histogrambetween the white edge and the ninety-seventh percentile). For example,the ninety-seventh percentile (e.g., the brightest 3% of pixels) in animage having 6.3 million pixels would encompass the 189,000 brightestpixels of the image. The distance to white 760, then, is the measurementof the distribution in the histogram of those 3% of the brightest pixelsof the image in some embodiments. For example, in a histogram of an8-bit JPEG image with 256 different values (e.g., luminance values from0-255), a sparse distribution of the 189,000 brightest pixels of thehistogram may span a portion of the histogram that covers the highest20% of histogram values (e.g., 204-255). The distance to white 760would, therefore, be a relatively large 20% of the histogram. On theother hand, a dense distribution of the 189,000 brightest pixels mayspan the highest 1.5% of histogram values (e.g., 252-255). Here, thedistance to white 760 would be a relatively small 1.5% of the histogram.While this example describes the specified percentage as 3% of the totalnumber of pixels for the image, other percentages (e.g., 10%, 0.1%,etc.) can be used in different embodiments.

In some embodiments, different curves are generated from the differentfunctions for tempering the adjustment of the target average luminance.The different functions, in some embodiments, are used to attenuate theamount of average luminance adjusted for different images havingdifferent distances to black and/or white. For example, for first andsecond images that have the same original average luminance value, butdifferent distances to black or white, a first function that is based onthe distance to black of a first histogram may specify a tempered targetluminance value for a first image, while a different second functionthat is based on the distance to black of a different second histogrammay specify a different tempered target luminance value from which amultiplier value is determined for adjusting the pixels of a secondimage.

In some embodiments, a particular function is selected from thedifferent functions based on the distance to black or the distance towhite in the histogram for the image. In these embodiments, the curvespecified by the function identifies the tempered target luminance valuefrom which a multiplier value is determined for adjusting all of thepixels of the image. In some embodiments, the desired adjustment (e.g.,increasing or decreasing luminance) is affected by only one of thedistance to black and the distance to white.

In some embodiments, when the average luminance for the pixels of theimage is low, the application automatically increases the luminance ofthe image. However, in some images, even though the average luminance islow, a significant minority of the pixels in the image may have highluminance. A large increase in the luminance value would cause many ofthe pixels to be blown out (e.g., increased beyond the maximum luminancevalue). Some embodiments increase the luminance less when there is asignificant minority of bright pixels than when there are relatively fewbright pixels.

Similarly, when the average luminance of the pixels is high, theapplication automatically decreases the luminance value. However, insome images, even though the average luminance is high, a significantminority of the pixels in the image have low luminance. A large decreasein the exposure value would cause a loss of visible detail among many ofthe darker pixels. Some embodiments decrease the luminance less whenthere is a significant minority of dark pixels than when there arerelatively few dark pixels.

Accordingly, in some of these embodiments, an adjustment to increase theaverage luminance of the image is affected by the distance to white,while an adjustment to decrease the average luminance is affected by thedistance to black. Thus, in some embodiments, an analysis of thedistance to black or distance to white for the histogram is needed todetermine the function to select. For example, a first image with a lotof pixels against the black edge of the histogram (i.e., a shortdistance to black) should not get much darker, while a second image withvery few (or no) pixels against the black edge of the histogram (i.e., along distance to black) may be darkened considerably. In this example,the function selected to adjust the average luminance of the first imagecould be a function that tempers the luminance adjustment more than thefunction selected to adjust the average luminance of the second image.

As each histogram has a different distribution of luminance values, theresults of decreasing brightness in the images represented by each ofthe histograms would be substantially different from each other. Asshown in FIG. 7, a first histogram 710 (Histogram A) charts thefrequency of pixels at different luminance values along the dynamicrange of the image. As shown, a majority of the pixels have luminancevalues that lie in a darker region of the histogram 710. As the majorityof pixels lie in the darker region of the histogram, the imagerepresented by the first histogram 710 has an overall exposure that ismore dark than it is bright. Some embodiments are equipped to adjust theexposure of images having such unbalanced exposure levels by identifyingthe distance to black and/or the distance to white. The distance toblack, for instance, is determined according to the particularpercentile specified for the darkest pixels (e.g., pixels having thelowest luminance values) of the image in some embodiments. In this case,a heavy distribution of pixels having luminance values in the darkerregion is reflected by the short white arrow indicating the distance toblack (D_(B)) 750. At the other end of the histogram 710, pixels aresparsely populated in a brighter region of the histogram. Thisrelatively light distribution of pixels in the brighter region isreflected by the long black arrow indicating the distance to white(D_(W)) 760. As above, the distance to white (D_(W)) 760 is determined,in some embodiments, according to the particular percentile specifiedfor the brightest pixels (e.g., pixels having the highest luminancevalues) of the image.

In some embodiments, the distance to black in the histogram isdetermined before reducing exposure brightness of the image bydecreasing the average luminance. In these embodiments, a temperingfunction is selected which optimizes the adjustment of the averageluminance for the image.

As shown for histogram A 710, in this example, the selected function hasthe shortest distance to black among the three histograms 710, 720, and730. In this case, the distance to black is short because a majority ofpixels are distributed in the darker region of the image. Therefore,there is little room to reduce the average luminance of the imagewithout affecting the darker pixels (e.g., reducing the brightness orluminance values of pixels may result in a number of dark pixels beingcrushed to the darkest black luminance value possible for the image).Thus, the selected function generates the tempering curve 756. The othertempering curves, 754 and 752, correlate different average luminancevalues for different images to different tempered target luminancevalues. In particular, the tempering curve 756 attenuates the luminanceadjustment more than the other tempering curves, 754 and 752. As shown,the luminance adjustments provided by the tempering curve 756 arefarther from the target luminance (horizontal dashed line) than eitherof the other tempering curves 754 and 752.

On the other hand, the distance to black for histogram B 720, in thisexample, is longer than the distance to black in histogram A 710, andshorter than the distance to black in histogram 730. Therefore, theselected function, in this example, generates the tempering curve 754.The distance to black for histogram C 730 is longer than the distancesto black for both histograms A and B 710 and 720. In this case, theselected function generates the tempering curve 752.

Similar to the determination of the distance to black for adjustmentsthat decrease the brightness of the image, in some embodiments thedistance to white is determined when increasing brightness for theimage. In some embodiments, the distance to white of the histogram isdetermined before increasing the average luminance of the image. Inthese embodiments, a tempering function is selected which optimizes theluminance adjustment for the image.

For instance, three different distances to white are shown in histogramsA, B, and C, which illustrate the selections of different temperingcurves. In particular, the distance to white in histogram A 710 is long,the distance to white in histogram C 730 is short, and the distance towhite in histogram B 720 is in the middle. In this example, the selectedfunction for increasing the average luminance in histogram A 710, wouldgenerate the tempering curve 766, the function for histogram B 720 wouldgenerate the tempering curve 764, and the function for histogram C 730would generate the tempering curve 762.

The examples described above generally discuss selecting a function fromseveral functions to temper the adjustments to luminance for modifyingthe exposure of the image. The next section describes a process foradjusting the average luminance of an image using multiple curves frommultiple functions.

B. Tempered Exposure Adjustment Process

FIG. 8 conceptually illustrates a process 800 of some embodiments thatadjusts the exposure of an image when the average luminance of the imageis not the same as a target luminance for the image. The auto-exposureadjustment module 410 of the media editing application performs theprocess 800, in some embodiments, when a user invokes the auto-exposureadjustment module 410. Moreover, the process 800 of some embodiments isperformed by the auto-exposure adjustment module 410 when anauto-enhance module of the media editing application invokes theauto-exposure adjustment module 410 during an auto-enhance pipelineprocess.

The process 800 begins by computing (at 810) a histogram of an image.The image may be retrieved from a storage (e.g., the original image 455retrieved from the image storage 450) or from a device that captured theimage (e.g., during an image import process of the media editingapplication).

Next, the process 800 identifies (at 820) the average luminance of theimage. In different embodiments, the average luminance expressesdifferent luminance values for the image. For instance, the averageluminance can be the arithmetic mean luminance, the median luminance, orany other measure of central tendency for the luminance of the image.

After identifying the average luminance of the image, the process 800determines (at 830) whether the original average luminance is the sameas a desired target luminance for the image. If the average luminance isthe same as the target luminance, the process 800 ends.

Otherwise, if the average luminance is not the same as the targetluminance, the process 800 proceeds to 840 to determine whether theoriginal average luminance is less than the target luminance. If theprocess determines (at 840) that the original average luminance is lessthan the target luminance, the process 800 identifies (at 850) thedistance to white. After identifying the distance to white, the process800 identifies (at 860) an auto-exposure adjustment multiplier, whichwhen applied to the original pixel values of the image, adjusts theexposure of the image (i.e., modifies the average luminance of theimage). The adjustment multiplier is a value by which the luminancevalue of every pixel in the image is multiplied to adjust the exposureof the image. In some embodiments, the adjustment multiplier is based onthe tempered target luminance value computed from the selected temperingfunction. After identifying the auto-exposure adjustment multiplier, theprocess 800 transitions to 890 to calculate the resulting luminancevalues of the pixels. The resulting average luminance of the image isset to equal the tempered target luminance value when the luminancevalue of every pixel in the image is multiplied by the adjustmentmultiplier.

Referring back to the determination made at 840, if the averageluminance is more than the target luminance, the process 800 proceeds to870 to identify a distance to black for the computed histogram. Afteridentifying the distance to black, the process 800 identifies (at 880)an auto-exposure adjustment multiplier (or divisor), which when appliedto the original pixel values of the image, adjusts the average luminanceof the image to modify the exposure. In some embodiments, the identifiedadjustment multiplier (or divisor) is based on the tempered targetluminance value computed from the selected tempering function. In someembodiments, the adjustment multiplier (or divisor) is a value by whichthe luminance value of every pixel in the image is multiplier (ordivided) to adjust the average luminance of the image, and therebymodify the exposure of the image. After identifying the auto-exposureadjustment multiplier (or divisor), the process 800 transitions to 890to calculate the resulting luminance values of the pixels of the image.In some embodiments, when the luminance value of every pixel in theimage is multiplied (or divided) by the adjustment multiplier (ordivisor), the average luminance of the image (i.e., the luminance valuesof all the pixels averaged) is equal to the tempered target luminancevalue.

Although the processes described above are described in a particularorder, different embodiments may perform these processes in a differentorder.

C. Different Curves for Different Distances to Black

Having discussed the selection of a tempering curve from multipletempering curves based on the distance to black or the distance towhite, the next several examples describe using different curves toadjust the average computed luminance in different histograms havingdifferent distances to black. Two examples are described by reference toFIG. 9, which depicts different curves for decreasing the averagecomputed luminance in two different image histograms having differentdistances to black.

i. Short Distance to Black

FIG. 9 illustrates two examples of decreasing the average computedluminance by using different tempering curves based on differentdistances to black for two different histograms having the same originalaverage computed luminance. FIG. 9 is described by reference to FIGS. 10and 11, which conceptually illustrate a graphical user interface (GUI)of a media editing application 1030 of some embodiments that adjusts theaverage computed luminance of an image. As shown, the media editingapplication 1030 has a set of user-selectable image editing controls1050 that includes a user-selectable auto exposure tool 1055 of someembodiments, a preview display area 1040 that displays images, and ahistogram display area 1060 that displays histograms of images displayedin the preview display area 1040.

In the first example of FIG. 9, which is described by reference to FIG.10, the tempering curve is based on a short distance to black. As shownin the first stage 1010 of FIG. 10, an original image is displayed inthe preview display area 1040 and a histogram 945 is displayed in thehistogram display area 1060 of the media editing application 1030. Theappearance of the original image at this stage 1010 is generally bright(e.g., sky, water, and person displayed in light tones) with only onesubstantially dark item displayed in the original image (e.g., thekayak). In some embodiments, the histogram 945 is computed when theoriginal image is loaded into the preview display area 1040. Forinstance, in the first stage 910 of the first example in FIG. 9, thehistogram 945 has been computed for the original image and representsthe distribution of luminance values for the pixels of the originalimage.

After the image histogram 945 is computed, in some embodiments, theintensity indicator 570 is set to the average computed luminance of theimage. In some embodiments, the average computed luminance is computedfrom the distribution of histogram values. In other embodiments, theaverage computed luminance is identified by retrieving (e.g., from astorage) a pre-computed average luminance value for the original image(e.g., pre-computed by the device that captured the original image orpre-computed during an image import process of the media editingapplication). As shown in the first stage 910 of the first example inFIG. 9, the intensity indicator 570 indicates that the average computedluminance is 0.8 for the original image.

In some embodiments, the media editing application 1030 automaticallyupdates one or more of the image editing controls 1050 of the GUI basedon the values (e.g., the average computed luminance) derived from theoriginal image histogram 945. For instance, in the first stage 1010 ofFIG. 10, the brightness slider 1053 automatically shows that the averagecomputed luminance of the first image is 0.8.

As shown in the first stage 1010, a user has selected the auto exposuretool 1055. In some embodiments, a selection of the auto exposure tool1055 causes the media editing application to evaluate the exposurecharacteristics of the original image to determine the type and amountof an exposure adjustment to perform. In some embodiments, thisevaluation is based on a pre-defined target luminance (e.g., 0.5) thatis desired for the image. In some embodiments, when the average computedluminance is greater than the target luminance, the media editingapplication performs a luminance reduction operation (e.g., darkens theimage). In contrast, when the average computed luminance is less thanthe target luminance, the media editing application increases theluminance of the image (e.g., brightens the image).

As shown in FIG. 9, the pre-defined target luminance is 0.5, which isless than the average computed luminance of 0.8 identified in the firststage 910 of FIG. 9. Accordingly, the media editing application, of someembodiments, performs an operation that reduces the average luminance ofthe image toward the target luminance desired for the image. In someembodiments, the operation for reducing the average luminance is basedon the distance to black in the histogram 945. In these embodiments,therefore, the media editing application identifies the distance toblack (e.g., determining the set black percentile and then identifyingthe location along the x-axis of the histogram at which the percentileresides). In this example, the identified distance to black is shortbecause, as shown at this stage 910, there is a significant spike ofpixels at the darkest luminance values of the histogram.

After identifying the distance to black, an exposure tempering curve isselected for tempering the average luminance adjustment. As describedabove, the media editing application 1030 of some embodiments selectsthe tempering curve from several different exposure tempering curvesbased on the identified distance to black. Then the selected temperingcurve is applied to determine the tempered luminance value. Shown in theresponse graph 950 at the second stage 920, the selected tempering curvecorrelates the average computed luminance value of 0.8 beforemodification to a modified average luminance value of 0.64.

For this figure, the tempered target luminance (T) is less than theoriginal average computed luminance (L). In order to lower the averageluminance of the image, the application of some embodiments multipliesthe luminance value of each pixel by a multiplier (M), which isdetermined by the equation:M=T/L  (9)

In some embodiments, the exposure adjustment value (EV) is calculatedbased on the multiplier, using the equation:EV=log₂(M)  (10)

The equations 9 and 10 lead to negative EVs when the tempered targetluminance (T) is less than the original average computed luminance (L).

As shown in the second stage 920, the pixel distribution in thehistogram 955 includes a taller and narrower spike of the darker pixelsand narrower and taller group of the brighter pixels, which are alsoshifted significantly to the left (i.e., darker). Furthermore, theintensity indicator 570 reflects the modified luminance value (0.64)based on the modified distribution of pixels toward the left (darker)side of the image histogram 955.

The modification to the average computed luminance of the original imageis shown in FIG. 10, where the modified image, displayed in the previewdisplay area 1040, appears darker than the image displayed in the firststage 1010. The histogram 955, representing this darker modified image,is also shown in the second stage 1020 of FIG. 10. As an updated set ofvalues are associated with this histogram 955, one or more of the imageediting tools 1050 (e.g., brightness, exposure) have been updated atthis stage 1020.

While this example discussed luminance modifications when the distanceto black is short, the next example discusses luminance modificationswhen the distance to black is long.

ii. Long Distance to Black

The second example of FIG. 9 is described by reference to FIG. 11 wherethe tempering curve is based on a long distance to black. As shown inthe first stage 1110 of FIG. 11, the histogram 965 is displayedrepresenting an original image (different from the original image ofFIG. 10) that is displayed in the preview display area 1040. Theappearance of the original image at this stage 1110 is generally brightwithout any substantially dark elements in the image.

As shown in the third stage 930 of FIG. 9, the intensity indicator 570indicates that the average computed luminance is 0.8 for the originalimage. Although the histogram 965 in this example is different from thehistogram 945 in the first example of FIG. 9, the average computedluminance for both histograms is the same.

As shown in the first stage 1110 of FIG. 11, the user has selected theauto exposure tool 1055. The pre-defined target luminance is 0.5, asshown in FIG. 9, is less than the average computed luminance of 0.8shown on the brightness slider 1053 in FIG. 11. Therefore, the averagecomputed luminance (0.8) is greater than the target luminance (0.5), sothe overall luminance will be reduced. In this example, the distance toblack 750 is large, so a tempering curve is selected from among severaltempering curves. The selected tempering curve at this stage 940 isdifferent from the tempering curve selected at stage 920. Shown in theresponse graph 970, the selected tempering curve maps the averagecomputed luminance value of 0.8 before modification to a modifiedaverage luminance value of 0.53. At this stage 940, the temperedluminance value is much closer to the target luminance value than thetempered luminance value of the second stage 920.

Once the tempered target luminance value is identified, the originalaverage computed luminance is reduced using the equations 9 and 10described above. However, in this example, the tempered target luminance(T) is closer to the actual target luminance value (e.g., 0.5) than itis to the original average computed luminance (L).

After the tempering function is applied and the pixel values arereduced, the histogram 975 shown at the fourth stage 940 exhibits themodified distribution of pixels (e.g., more spread out than the nearlyone-sided distribution of pixels at the third stage 930). The intensityindicator 570 also reflects the modified average luminance value at thetempered target value (0.53). The modification to the average computedluminance of the original image is shown in FIG. 11, where the modifiedimage appears darker than the original image.

D. Different Curves for Different Distances to White

Having described some examples of using different tempering curves basedon different histograms having different distances to black, the nextsection describes using different tempering curves based on differenthistograms having different distances to white. The next two examplesare described by reference to FIG. 12, which depicts different curvesfor increasing luminance in two different image histograms havingdifferent distances to white.

i. Short Distance to White

The first example of FIG. 12 is described by reference to FIG. 13, inwhich the tempering curve used to adjust the exposure of an image isbased on a short distance to white. As shown in the first stage 1210 ofFIG. 12, the histogram 1245 represents a distribution of image pixelvalues (i.e., luminance values) for an original image. In this example,the histogram represents the image shown in the preview display area1040 in FIG. 13. The appearance of the original image in the previewdisplay area 1040 is generally dark (e.g., the house, the door, and theroof are displayed in dark tones) with only a few bright image features(e.g., the windows on the house, the sky). At this stage 1210, apre-defined target luminance (0.5) is shown in the response graph 1240and the intensity indicator 570 indicates that the average computedluminance for the original image is 0.25.

In the first stage 1310 of FIG. 13, the user selects the auto exposuretool 1055. In this example, the auto exposure adjustment operationincreases luminance because the target luminance (0.5) is more than theaverage computed luminance (0.25) shown on the brightness slider 1053 inFIG. 13. The amount of increase, in this example, is based on a shortdistance to white 760, as shown in the histogram 1245. Accordingly, atempering curve for this short distance to white is selected from amongseveral tempering curves.

In the second stage 1220, the selected tempering curve is displayed inthe response graph 1250. Based on the selected tempering curve, theaverage computed luminance value of 0.25 is correlated to a temperedluminance value of 0.38. Once the tempered luminance value isidentified, the image is brightened using a multiplier, such as the onedetermined using equation 9. Then, equation 10 is used to determine theexposure value (i.e., EV=log₂(M)) to increase the image luminance. Atthe second stage 1220, the histogram 1255 represents the modifieddistribution of pixels after the tempering function is applied. Theintensity indicator 570 also represents the modified luminance value(0.38) at this stage 1220.

In the second stage 1320 of FIG. 13, the modified average luminance isshown in the GUI 1030 by the new position of the brightness slider 1053and the value (0.38) corresponding to the position of the brightnessslider 1053. As shown in this figure, the image displayed in the previewdisplay area is visibly brighter than the image displayed in the previewdisplay area 1040 during the first stage 1310. Furthermore, notablefeatures of the image (e.g., the windows panes) remain visible after theauto exposure adjustment in the second stage 1320.

ii. Long Distance to White

The second example of FIG. 12 is described by reference to FIG. 14 wherethe tempering curve is based on a long distance to white. As shown inthe third stage 1230 of FIG. 12, the histogram 1265 represents anoriginal image that is shown in the preview display area 1040 in FIG.14. The appearance of the original image at this stage 1210 is generallydark. The intensity indicator 570 indicates that the average computedluminance is 0.25 for the original image. Although the histogram 1265 inthis example is different from the histogram 1245 in the first exampleof FIG. 12, the average computed luminance for both histograms is thesame.

In the first stage 1410 of FIG. 14, the user selects the auto exposuretool 1055. The pre-defined target luminance is 0.5, as shown in FIG. 12,is more than the average computed luminance of 0.25 in this example, andtherefore, the luminance will be increased. In this example, a differenttempering curve is selected from among several tempering curves based ona large distance to white 760, as opposed to the short distance to white750 shown in the second stage 1220. Also, the response graph 1270displays the selected tempering curve, which maps the average computedluminance value of 0.25 to a tempered luminance value of 0.46. At thisstage 1240, the tempered luminance value is much closer to the targetluminance value than the tempered luminance value of the second stage1220.

Once the tempered luminance value is identified, the original averagecomputed luminance is increased by the multiplier. The intensityindicator 570 now indicates the modified luminance value (0.46), and thebrighter image displayed at the second stage 1420 of FIG. 14 reflectsthe increased average luminance value.

E. Exposure Adjustment Comparisons

Having described some examples of using different tempering curves basedon different histograms having different distances to black anddifferent distances to white, the next section compares full exposureadjustments with tempered exposure adjustments. In particular, fullexposure adjustments include adjusting the original average computedluminance of the image all the way to the target luminance, whiletempered exposure adjustments include tempering the adjustment to onlyadjust the original average computed luminance of the image to thetempered target luminance value.

i. Short Distance to White Comparison

FIG. 15 conceptually illustrates two examples that illustrate thedifference between a full exposure adjustment of an original image and atempered exposure adjustment of the image. In these examples, theaverage computed luminance of the original image is increased (e.g.,brightened) when the distance to white is short. The GUI illustrated inFIG. 15 is similar to the GUI 1030 shown in FIG. 13, however, in FIG.15, the average computed luminance of the original image is fullyadjusted to the target luminance value in the first example, while theaverage computed luminance of the original image is adjusted to thetempered luminance value in the second example.

The first stage 1510 shows the GUI with the original image displayed inthe preview display area 1040 and histogram 1345 displayed in thehistogram display area 1060. The set of exposure adjustment tools 1050are shown at this stage with exposure settings based on an analysis ofthe histogram 1345. As shown at this stage, a user selects the autoexposure tool 1055. In some embodiments, the auto exposure adjustmenttool 1055 modifies the original average computed luminance of the imageall the way to the pre-determined target luminance value. In theseembodiments, the auto exposure adjustment tool 1055 does not temper theexposure adjustment.

The second stage 1520 shows the image after the auto exposureadjustment. As shown in the GUI, the image appears substantiallybrighter than in the first stage 1510. The brightness of the image atthis stage 1520 is reflected in the histogram 1565. As shown, thedistribution of pixels in the histogram 1565 includes a heavyconcentration of pixels with values at the upper (i.e., brighter)values. Accordingly, several features of the image appear blown out orindistinguishable at this stage 1520. For instance, the windows on thefront of the house do not show window panes, and the borders of thewindows are hard to distinguish from the front wall of the house. Thus,the full exposure adjustment to the target exposure value has brightenedthe image to a degree that certain details of the image have been lostor blown out.

In contrast, the third stage 1530 shows the image and histogram 1355after the tempered auto exposure adjustment. In this example, thetempered luminance adjustment of the original image's average computedluminance results in a slightly brighter image. Furthermore, details andfeatures of the image have been retained (e.g., window panes arevisible, windows are distinct from the front wall of the house, etc.).

ii. Short Distance to Black Comparison

FIG. 16 conceptually illustrates two examples that illustrate thedifference between a full exposure adjustment of an original image and atempered exposure adjustment of the image. In these examples, theoriginal image is darkened when the distance to black is short. The GUIillustrated in FIG. 16 is similar to the GUI 1030 shown in FIG. 10,however, in FIG. 16, the average computed luminance of the originalimage is fully adjusted to the target luminance value in the firstexample, while the average computed luminance of the original image isadjusted to a tempered luminance value in the second example.

The first stage 1610 shows the GUI with the original image andhistogram. A user selects the auto exposure tool 1055 at this stage 1610to perform an automatic exposure adjustment.

The second stage 1620 shows a substantially darker appearance of theimage after the average computed luminance of the image was adjusted bythe auto exposure adjustment. As shown in the histogram 1655 at thisstage 1620, a very large portion of the pixels are located at lower(i.e., darker) values. Several features of the image appear dark withlittle variation in the level of darkness. Thus, the full adjustment ofthe average computed luminance of the original image to the targetluminance value has darkened the image to a degree that certain detailsof the image have been crushed or have become indistinguishable fromother details.

In contrast, the third stage 1630 shows the image after the temperedauto exposure adjustment modifies the average computed luminance of theoriginal image. In this case, the auto exposure adjustment results inonly a slightly darker image, while key image features and detailsremain distinct from other features and elements of the image.

As these examples show, the automatic exposure adjustments to theaverage computed luminance of the original image the fully modify theaverage luminance to the desired target luminance value, in some cases,result in loss of visible details in the image. Such distortion isalleviated by using the tempered exposure adjustments of the examplesdescribed above.

While the applications of some embodiments described previouslycalculate a tempered adjustment to move either the average intensity orthe median intensity (the 50^(th) percentile intensity) toward a targetvalue, the applications of other embodiments move a different percentile(e.g., the 18^(th) percentile) toward a target value.

IV. Electronic System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or morecomputational or processing unit(s) (e.g., one or more processors, coresof processors, or other processing units), they cause the processingunit(s) to perform the actions indicated in the instructions. Examplesof computer readable media include, but are not limited to, CD-ROMs,flash drives, random access memory (RAM) chips, hard drives, erasableprogrammable read-only memories (EPROMs), electrically erasableprogrammable read-only memories (EEPROMs), etc. The computer readablemedia does not include carrier waves and electronic signals passingwirelessly or over wired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storagewhich can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

FIG. 17 conceptually illustrates an example of an electronic system 1700with which some embodiments are implemented. The electronic system 1700may be a computer (e.g., a desktop computer, personal computer, tabletcomputer, etc.), phone, PDA, or any other sort of electronic orcomputing device. Such an electronic system includes various types ofcomputer readable media and interfaces for various other types ofcomputer readable media. Electronic system 1700 includes a bus 1705,processing unit(s) 1710, a graphics processing unit (GPU) 1715, a systemmemory 1720, a network 1725, a read-only memory 1730, a permanentstorage device 1735, input devices 1740, and output devices 1745.

The bus 1705 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 1700. For instance, the bus 1705 communicativelyconnects the processing unit(s) 1710 with the read-only memory 1730, theGPU 1715, the system memory 1720, and the permanent storage device 1735.

From these various memory units, the processing unit(s) 1710 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Someinstructions are passed to and executed by the GPU 1715. The GPU 1715can offload various computations or complement the image processingprovided by the processing unit(s) 1710. In some embodiments, suchfunctionality can be provided using CoreImage's kernel shading language.

The read-only-memory (ROM) 1730 stores static data and instructions thatare needed by the processing unit(s) 1710 and other modules of theelectronic system. The permanent storage device 1735, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system1700 is off. Some embodiments use a mass-storage device (such as amagnetic or optical disk and its corresponding disk drive) as thepermanent storage device 1735.

Other embodiments use a removable storage device (such as a floppy disk,flash memory device, etc., and its corresponding drive) as the permanentstorage device. Like the permanent storage device 1735, the systemmemory 1720 is a read-and-write memory device. However, unlike storagedevice 1735, the system memory 1720 is a volatile read-and-write memory,such a random access memory. The system memory 1720 stores some of theinstructions and data that the processor needs at runtime. In someembodiments, the invention's processes are stored in the system memory1720, the permanent storage device 1735, and/or the read-only memory1730. For example, the various memory units include instructions forprocessing multimedia clips in accordance with some embodiments. Fromthese various memory units, the processing unit(s) 1710 retrievesinstructions to execute and data to process in order to execute theprocesses of some embodiments.

The bus 1705 also connects to the input and output devices 1740 and1745. The input devices 1740 enable the user to communicate informationand select commands to the electronic system. The input devices 1740include alphanumeric keyboards and pointing devices (also called “cursorcontrol devices”), cameras (e.g., webcams), microphones or similardevices for receiving voice commands, etc. The output devices 1745display images generated by the electronic system or otherwise outputdata. The output devices 1745 include printers and display devices, suchas cathode ray tubes (CRT) or liquid crystal displays (LCD), as well asspeakers or similar audio output devices. Some embodiments includedevices such as a touchscreen that function as both input and outputdevices.

Finally, as shown in FIG. 17, bus 1705 also couples electronic system1700 to a network 1725 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or anIntranet), or a network of networks, such as the Internet. Any or allcomponents of electronic system 1700 may be used in conjunction with theinvention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself. In addition, someembodiments execute software stored in programmable logic devices(PLDs), ROM, or RAM devices.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium,” “computer readable media,” and “machinereadable medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by a computer. Theseterms exclude any wireless signals, wired download signals, and anyother ephemeral signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. For instance, many of the figuresillustrate various touch gestures (e.g., taps, double taps, swipegestures, press and hold gestures, etc.). However, many of theillustrated operations could be performed via different touch gestures(e.g., a swipe instead of a tap, etc.) or by non-touch input (e.g.,using a cursor controller, a keyboard, a touchpad/trackpad, a near-touchsensitive screen, etc.). In addition, a number of the figures (includingFIGS. 1 and 8) conceptually illustrate processes. The specificoperations of these processes may not be performed in the exact ordershown and described. The specific operations may not be performed in onecontinuous series of operations, and different specific operations maybe performed in different embodiments. Furthermore, the process could beimplemented using several sub-processes, or as part of a larger macroprocess. Thus, one of ordinary skill in the art would understand thatthe invention is not to be limited by the foregoing illustrativedetails, but rather is to be defined by the appended claims.

The invention claimed is:
 1. For an image editing application, a methodfor performing an image editing operation on an image based on a set ofimage processing operations performed by an image capturing device thatcaptured the image, the method comprising: at the image editingapplication, receiving an image adjusted by the set of image processingoperations performed by the image capturing device, that captured theimage, prior to importing the image into the image editing application;identifying a first set of parameters that quantify a first set ofadjustments made to the image by the set of image processing operationsperformed by the image capturing device; computing a second set ofparameters that quantify a desired second set of adjustments to theimage based on an analysis of the image properties; deriving a third setof parameters by comparing the first and second sets of parameters; andbased on the third set of parameters, performing the image editingoperation on the image to produce an adjusted image.
 2. The method ofclaim 1, wherein the image is an image of a scene captured by the imagecapturing device.
 3. The method of claim 1, wherein the first set ofparameters quantify a set of pixel intensities of the image.
 4. Themethod of claim 1, wherein the set of image processing operations adjustintensity of the image.
 5. The method of claim 1, wherein identifyingthe first set of parameters comprises computing an average luminance ofthe image.
 6. An electronic device comprising: a set of processingunits; a non-transitory machine readable medium storing a media editingapplication for execution by at least one processing unit of the device,the media editing application having a graphical user interface (GUI),the media editing application comprising sets of instructions for:receiving input to automatically adjust a parameter of an imagedisplayed in a GUI, the image having an appearance based on an initialvalue for the parameter; comparing the initial value for the parameterto a prespecified target value for the parameter; based on a differencebetween the initial value and the prespecified target value, identifyinga tempered target value for the parameter, wherein the tempered targetvalue is specific to the image and different than the prespecifiedtarget value; automatically adjusting the initial value for theparameter of the image to match the tempered target value rather thanthe prespecified target value for the parameter; and modifying theappearance of the displayed image based on the adjusted value of theparameter.
 7. The device of claim 6, wherein the initial value for theparameter is based on a set of image processing operations performed byan image capturing device that captured a scene to produce the image. 8.The device of claim 6, wherein the media editing application furthercomprises a set of instructions for importing the image into the mediaediting application, wherein the initial value for the parameter isbased on a set of image processing operations performed while importingthe image.
 9. The device of claim 6, wherein the initial value for theparameter comprises a set of image values based on a histogram of theimage.
 10. The device of claim 6, wherein the image comprises aplurality of pixels each of which has an associated luminance value,wherein the media editing application further comprises a set ofinstructions for computing an average luminance of the image based onthe luminance values of the pixels.
 11. The device of claim 10, whereinthe initial value for the parameter comprises the average luminance, theprespecified target value for the parameter comprises a target averageluminance, the tempered target value comprises a tempered averageluminance based on comparing the computed average luminance to thetarget average luminance, and automatically adjusting the initial valuefor the parameter comprises adjusting the computed average luminance tothe tempered average luminance.
 12. The device of claim 6, wherein themedia editing application further comprises a set of instructions fordisplaying a selectable tool for automatically adjusting the parameter,wherein the set of instructions for receiving input to automaticallyadjust the parameter comprises a set of instructions for receiving aselection of the tool.
 13. A non-transitory machine readable mediumstoring an image editing application which when executed by at least oneprocessing unit performs an image editing operation on an image based ona set of image processing operations performed by an image capturingdevice that captured the image, the program comprising sets ofinstructions for: at the image editing application, receiving an imageadjusted by the set of image processing operations performed by theimage capturing device, that captured the image, prior to importing theimage into the image editing application; identifying a first set ofparameters that quantify a first set of adjustments made to the image bythe set of image processing operations performed by the image capturingdevice; computing a second set of parameters that quantify a desiredsecond set of adjustments to the image based on an analysis of the imageproperties; deriving a third set of parameters by comparing the firstand second sets of parameters; and based on the third set of parameters,performing the image editing operation on the image to produce anadjusted image.
 14. The non-transitory machine readable medium of claim13, wherein the image is an image of a scene captured by the imagecapturing device.
 15. The non-transitory machine readable medium ofclaim 13, wherein the first set of parameters quantify a set of pixelintensities of the image.
 16. The non-transitory machine readable mediumof claim 13, wherein the set of image processing operations adjustintensity of the image.
 17. The non-transitory machine readable mediumof claim 13, wherein the set of instructions for identifying the firstset of parameters comprises a set of instructions for computing anaverage luminance of the image.
 18. For a media-editing applicationhaving a graphical user interface (GUI), a method comprising: receivinginput to automatically adjust a parameter of an image displayed in aGUI, the image having an appearance based on an initial value for theparameter; comparing the initial value for the parameter to aprespecified target value for the parameter; based on a differencebetween the initial value and the prespecified target value, identifyinga tempered target value for the parameter, wherein the tempered targetvalue is specific to the image and different than the prespecifiedtarget value; automatically adjusting the initial value for theparameter of the image to match the tempered target value rather thanthe prespecified target value for the parameter; and modifying theappearance of the displayed image based on the adjusted value of theparameter.
 19. The method of claim 18, wherein the initial value for theparameter is based on a set of image processing operations performed byan image capturing device that captured a scene to produce the image.20. The method of claim 18 further comprising importing the image intothe media editing application, wherein the initial value for theparameter is based on a set of image processing operations performedwhile importing the image.
 21. The method of claim 18, wherein theinitial value for the parameter comprises a set of image values based ona histogram of the image.
 22. The method of claim 18, wherein the imagecomprises a plurality of pixels each of which has an associatedluminance value, the method further comprising computing an averageluminance of the image based on the luminance values of the pixels. 23.The method of claim 22, wherein the initial value for the parametercomprises the average luminance, the prespecified target value for theparameter comprises a target average luminance, the tempered targetvalue comprises a tempered average luminance based on comparing thecomputed average luminance to the target average luminance, andautomatically adjusting the initial value for the comprises adjustingthe computed average luminance to the tempered average luminance. 24.The method of claim 18 further comprising displaying a selectable toolfor automatically adjusting the parameter, wherein receiving input toautomatically adjust the parameter comprises receiving a selection ofthe tool.